Adequately testing a program involves exercising the function of that program under conditions which reflect the environment in which the program was designed to operate. For programs designed to operate on a single computer, testing requires use of actual configurations of machines on which the program was to be run (or a simulation of such configuration) and test data which reflected the range of possible inputs to the program. For programs larger in scale, consisting of multiple subsystems, each subsystem itself composed of multiple modules, adequately testing the whole involves testing each of the modules, integrating the modules into their respective subsystems, testing the subsystem, and finally testing the entire system.
Testing in a distributed environment presents additional problems. A distributed environment is one composed of multiple computer systems operating independently and connected by a common transmission line over which they communicate with each other. Testing a program in the same hardware configuration and operating conditions becomes extremely difficult. (For a discussion of the problems see Mori, et al., Method and Apparatus for Testing a Distributed Computer System, U.S. Pat. No. 4,803,683.) In addition, in the prior art, testing in a distributed environment commonly involved multiple test teams, each working at one of the distributed systems. When changes were made to application programs, testing the changes involved distributing the changed program and associated test cases and coordinating the results of each test team. Thus, considerable effort was required simply to maintain test case libraries and track test case execution status.
Further complicating the problems related to adequately reflecting the operational environment, proper testing in a distributed environment requires precise timing and sequencing of test events, in order to produce consistent, checkable results. This has been difficult, even impossible to achieve using humans and standalone tools. Another challenge to testers is the task of tracking testcase status across multiple sets of distributed systems, perhaps including several different versions of the software system, has proven to be difficult and time-consuming.